Parallelization of Irregular Codes Including Out-of-Core Data and Index Arrays

This paper describes techniques for implementing irregular out-of-core codes on distributed memory machines. These codes involve data arrays and other data structures that are too large to t in main memory; so data needs to be stored on disks and fetched during the execution of the program. The eecient use of disk storage is a critical factor that determines the performance of these applications. We propose several program transformations and optimization techniques and a runtime system support to reduce the I/O and communication overheads for out-of-core programs. The eeciency of our prototype implementation of these techniques is demonstrated by experimental results taken from a template CFD code.